The Artificial Intelligence Laboratory and The Research Laboratory of Electronics at

In this paper, we summarize our progress on two interaction techniques (point-based and ray-based) for haptic rendering. First, we describe a new point-based rendering technique that is computationally less expensive than previously proposed methods. We use a localized search procedure to reduce the number of computations and make it independent of the number polygons of the polyhedron. Second, we discuss the implementation details of a ray-based technique together with its advantages. We have utilized this rendering technique to display the haptic interactions between 3D objects and the stylus modeled as a line segment. In order to develop more effective multimodal VEs, we have experimented with multi-threading (on Windows NT platform) and multi-processing (on UNIX platform) techniques and have successfully separated the visual and haptic servo loops. Our experience is that both techniques are quite useful in achieving high haptic rendering rates and stable haptic interactions. Although creating a separate process for each modality requires more programming effort, it enables the user to display the graphics and/or haptics on any desired machine(s). On the other hand, programming with threads takes less effort, but they are not as flexible as processes. We have also developed a graphical interface that enables the user to construct virtual environments by means of user-defined text file, toggle stereo visualization, save the virtual environment and quit from the application. The user can load objects into the scene, assign simple visual and haptic properties to the objects using this text file. Following the construction of the scene using the text file, the user can interactively translate, rotate, and scale objects and the interface will automatically update the visual and haptic models. Collision Detection In haptic rendering, one of the important problems is the detection of collisions between the end-effector (or generic probe) of the haptic device and the 3D objects in the scene. A good collision detection algorithm not only increases the servo rate, but also helps in correctly displaying interaction forces to the human operator to make the virtual objects more realistic. The collision detection techniques used in haptic rendering slightly differ from the ones used in computer graphics. In computer graphics, the collision detection is used to detect if two objects overlap. On the other hand, the purpose of the collision detection in haptic rendering is not only to check collisions between objects, but also to calculate the appropriate interaction forces to convey to the user a tactual sensation of 3D objects. The calculated interaction force increases with the penetration depth and has the effect of preventing further penetration of stylus into object surfaces. We propose two types of haptic interaction algorithms here: point-based and ray-based. For the pointbased haptic interaction algorithm, the probe is simply modeled as a point. For the ray-based collision detection algorithm, the probe is modeled as a line segment. Both techniques have advantages and disadvantages. For example, it is computationally less expensive to render 3D objects using point-based technique. Hence, we achieve higher haptic servo rates. On the other hand, the ray-based haptic interaction technique handles side collisions and can provide additional haptic cues for conveying to the user the shape of objects. ·I_ ___ _^_ 1_ ·111_ _ __CI__ II1I -----1I1I I CII-··---· --·--· ·-III1 ·C·IY·IPU --I-Point-Based Haptic Interaction Algorithm The point-based haptic interaction algorithm we have proposed is similar to the god-object algorithm (Zilles and Salisbury, 1995) but follows a different approach. We believe that our approach increases the servo rate, makes the haptic interactions more stable and the servo rate independent of the number of polygons. In this approach, the first step is to construct a hierarchical data base for the geometrical properties of the 3D objects. The data base is used to store geometrical information of the polygonal objects in the virtual environment. In the data base, we assume that the polyhedrons are made of three types of geometrical primitives: polygon, line, and vertex. Each polygon has neighboring primitives of lines and vertices. Each line has neighboring primitives of polygons and vertices, and each vertex has neighboring primitives of polygons and lines. In the data base, each primitive also has a normal vector attached to it. For a polygon primitive, the normal is the vector that is perpendicular to its surface and points outside. For a vertex or line primitive, its normal is the average of the normals of its neighboring polygons. When exploring the virtual environments, we interact with objects through the Haptic Interface Point (HIP). At the same time, we also consider another point, ideal haptic interface point (IHIP) -same as the so called god-object(Zilles and Salisbury, 1995)to follow the trace of the HIP. The HIP is not constrained, and because haptic interface devices have a finite maximum stiffness, it can penetrate the object. However, we can constrain the IHIP such that it doesn't penetrate any object. When the HIP is outside the virtual object, the IHIP will be coincident with the HIP. When the probe penetrates an object, the IHIP will stay on the surface of the object. When the user moves the HIP by manipulating the probe, the IHIP will also be moved to a new position depending on the situation. The rule for moving the IHIP is that the IHIP will move towards the probe and cannot penetrate any surface. Furthermore, we assume that there is a virtual spring between the HIP and the IHIP. The force applied to the HIP is calculated based on the distance between the HIP and the IHIP. When the HIP is outside the object, we keep track of its path and check if this path penetrates any polygon surface. We construct a line segment from the previous (tail) and the current (tip) coordinates of the HIP and detect the collisions between the line segment and the polygons of the 3D object. If this line segment penetrates the object, then we choose the surface that is nearest to the previous HIP as the contacted geometric primitive. The IHIP will then stay on this surface. We then calculate the nearest distance from the current HIP to the contacted primitive and its neighboring primitives (We only check the contacted primitive and its neighboring primitives, not all of the primitives. For example, if the contacted primitive is polygon, then we check the distance from current HIP to the neighboring lines and vertices. This approach significantly reduce the number of computations). Finally, we set the primitive that has the shortest distance to the HIP (the priority of selecting the nearest primitive goes as vertex, line, and polygon) as the contacted primitive and move the IHIP to the point that is on this primitive and nearest to the HIP. After setting the new position of IHIP, we construct a vector from the IHIP to HIP. If this vector and the normal of the contacted primitive are in the same direction, the HIP is no longer inside the object. if (collision = = FALSE) primitivel = the contacted primitive of the previous loop; if (the path of HIP penetrates a polygon) distancel = nearest distance from current HIP to { primitivel 1. set the polygon as the contacted primitive for (i=l: number of neighboring primitives of 2. move the IHIP to this surface primitivel) } primitive2 = the ih neighboring primitive of else // there is a collision primitivel { distance2 = distance from HIP to primitive2 if(distance2 < distance l) Vectorl = vector from IHIP to HIP; Normal I = normal of the contacted primitive; contacted primitive = primitive2; if( dot(Vectorl, Normall) > 0) distancel = distance2; collision = FALSE; else i collision = TRUE; move IHIP to the point that is nearest from the contacted primitive to HIP; Ray-Based Haptic Interaction Algorithm The ray-based haptic rendering is a haptic interaction technique we have developed recently (Basdogan et. al, 1997, Srinivasan and Basdogan, 1997). In ray-based rendering, we model the generic probe of the haptic device as a line segment and detect the collisions with 3D objects in the scene. Therefore, the collision detection will be much more complicated than that in point-based case. In the point-based case, the only possible collision situations are point-point, point-line, and point-polygon collisions. However, the collisions in the ray-based case can in addition be line-point, line-line, line-polygon collisions. There can also be multiple contacts composed of a combination of the above cases. Since the collisions in ray-based are more complicated, it is not easy to develop a set of simple rules that can handle all of the cases. Therefore, we opted to develop a rule-based algorithm that can successfully handle only the collision conditions between a line probe and convex objects. However, this is not a major limitation since the concave polyhedron objects can always be divided into several convex objects. The raybased collision detection algorithm that we propose follows a two-stage approach for detecting the collisions. Analogous to IHIP in point-based interactions, we consider another stylus (we call it the ideal stylus) to follow the trace of the simulated stylus. Similar to the HIP, the simulated stylus can penetrate the objects, but the ideal stylus is constrained to stay on the surface of the object and to be always parallel to the simulated stylus. We assume that there are two springs between the simulated stylus and the ideal stylus one between the tips and the other between the tails, respectively. This arrangement will help us in computing torque in addition to the forces to be displayed to the users. It should be noted here that the only plausible collision conditions are point-polygon and line-line contacts because we consider only convex objects. The point-point, point-line, line-point, and line-polygon are all unlikely to occur, because at best they occur in a single servo loop. point-polygon contact line-line contact Figure 1. Plausible collision conditions between the simulated stylus and convex object in ray-based interactions. When implementing this algorithm, the main challenge is to correctly position the ideal stylus. We use twostage movement technique to move the ideal stylus.